<!DOCTYPE html>


<html lang="zh-CN">


<head>
  <meta charset="utf-8" />
    
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" />
  <title>
    Ceph分布式存储日常运维管理.md |  
  </title>
  <meta name="generator" content="hexo-theme-ayer">
  
  <link rel="shortcut icon" href="/favicon.ico" />
  
  
<link rel="stylesheet" href="/dist/main.css">

  
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/Shen-Yu/cdn/css/remixicon.min.css">

  
<link rel="stylesheet" href="/css/custom.css">

  
  
<script src="https://cdn.jsdelivr.net/npm/pace-js@1.0.2/pace.min.js"></script>

  
  

  

</head>

</html>

<body>
  <div id="app">
    
      
    <main class="content on">
      <section class="outer">
  <article
  id="post-deploy/Ceph分布式存储日常运维管理"
  class="article article-type-post"
  itemscope
  itemprop="blogPost"
  data-scroll-reveal
>
  <div class="article-inner">
    
    <header class="article-header">
       
<h1 class="article-title sea-center" style="border-left:0" itemprop="name">
  Ceph分布式存储日常运维管理.md
</h1>
 

    </header>
     
    <div class="article-meta">
      <a href="/2020/11/11/deploy/Ceph%E5%88%86%E5%B8%83%E5%BC%8F%E5%AD%98%E5%82%A8%E6%97%A5%E5%B8%B8%E8%BF%90%E7%BB%B4%E7%AE%A1%E7%90%86/" class="article-date">
  <time datetime="2020-11-10T16:00:00.000Z" itemprop="datePublished">2020-11-11</time>
</a> 
  <div class="article-category">
    <a class="article-category-link" href="/categories/deploy/">deploy</a>
  </div>
  
<div class="word_count">
    <span class="post-time">
        <span class="post-meta-item-icon">
            <i class="ri-quill-pen-line"></i>
            <span class="post-meta-item-text"> 字数统计:</span>
            <span class="post-count">3.1k</span>
        </span>
    </span>

    <span class="post-time">
        &nbsp; | &nbsp;
        <span class="post-meta-item-icon">
            <i class="ri-book-open-line"></i>
            <span class="post-meta-item-text"> 阅读时长≈</span>
            <span class="post-count">12 分钟</span>
        </span>
    </span>
</div>
 
    </div>
      
    <div class="tocbot"></div>




  
    <div class="article-entry" itemprop="articleBody">
       
  <h1 id="Ceph分布式存储日常运维管理"><a href="#Ceph分布式存储日常运维管理" class="headerlink" title="Ceph分布式存储日常运维管理"></a>Ceph分布式存储日常运维管理</h1><blockquote>
<p>作者：过眼风</p>
<p>来源：<a target="_blank" rel="noopener" href="https://www.cnblogs.com/yuezhimi">https://www.cnblogs.com/yuezhimi</a></p>
</blockquote>
<p><img src="http://iubest.gitee.io/pic/640-1601174189983.png" alt="img"></p>
<h3 id="常见问题"><a href="#常见问题" class="headerlink" title="常见问题"></a>常见问题</h3><p>nearfull osd(s) or pool(s) nearfull 此时说明部分osd的存储已经超过阈值，mon会监控ceph集群中OSD空间使用情况。如果要消除WARN,可以修改这两个参数，提高阈值，但是通过实践发现并不能解决问题，可以通过观察osd的数据分布情况来分析原因。</p>
<p>配置文件设置阈值</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">&quot;mon_osd_full_ratio&quot;:&quot;0.95&quot;,</span><br><span class="line">&quot;mon_osd_nearfull_ratio&quot;:&quot;0.85&quot;</span><br></pre></td></tr></table></figure>

<p>自动处理</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">ceph osd reweight-by-utilization</span><br><span class="line">ceph osd reweight-by-pg 105 cephfs_data(pool_name)</span><br></pre></td></tr></table></figure>

<p>手动处理</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ceph osd reweight osd.20.8</span><br></pre></td></tr></table></figure>

<p>全局处理</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">ceph mgr module ls</span><br><span class="line">ceph mgr module enable balancer</span><br><span class="line">ceph balancer on</span><br><span class="line">ceph balancer mode crush-compat</span><br><span class="line">ceph config-key set&quot;mgr&#x2F;balancer&#x2F;max_misplaced&quot;:&quot;0.01&quot;</span><br></pre></td></tr></table></figure>

<h3 id="PG-故障状态"><a href="#PG-故障状态" class="headerlink" title="PG 故障状态"></a>PG 故障状态</h3><p><strong>PG状态概述 一个PG在它的生命周期的不同时刻可能会处于以下几种状态中:</strong></p>
<ol>
<li>Creating(创建中) 在创建POOL时,需要指定PG的数量,此时PG的状态便处于creating,意思是Ceph正在创建PG。</li>
<li>Peering(互联中) peering的作用主要是在PG及其副本所在的OSD之间建立互联,并使得OSD之间就这些PG中的object及其元数据达成一致。</li>
<li>Active(活跃的) 处于该状态意味着数据已经完好的保存到了主PG及副本PG中,并且Ceph已经完成了peering工作。</li>
<li>Clean(整洁的) 当某个PG处于clean状态时,则说明对应的主OSD及副本OSD已经成功互联,并且没有偏离的PG。也意味着Ceph已经将该PG中的对象按照规定的副本数进行了复制操作。</li>
<li>Degraded(降级的) 当某个PG的副本数未达到规定个数时,该PG便处于degraded状态,例如:</li>
<li>在客户端向主OSD写入object的过程,object的副本是由主OSD负责向副本OSD写入的,直到副本OSD在创建object副本完成,并向主OSD发出完成信息前,该PG的状态都会一直处于degraded状态。又或者是某个OSD的状态变成了down,那么该OSD上的所有PG都会被标记为degraded。当Ceph因为某些原因无法找到某个PG内的一个或多个object时,该PG也会被标记为degraded状态。此时客户端不能读写找不到的对象,但是仍然能访问位于该PG内的其他object。</li>
<li>Recovering(恢复中) 当某个OSD因为某些原因down了,该OSD内PG的object会落后于它所对应的PG副本。而在该OSD重新up之后,该OSD中的内容必须更新到当前状态,处于此过程中的PG状态便是recovering。</li>
<li>Backfilling(回填) 当有新的OSD加入集群时,CRUSH会把现有集群内的部分PG分配给它。这些被重新分配到新OSD的PG状态便处于backfilling。</li>
<li>Remapped(重映射) 当负责维护某个PG的acting set变更时,PG需要从原来的acting set迁移至新的acting set。这个过程需要一段时间,所以在此期间,相关PG的状态便会标记为remapped。</li>
<li>Stale(陈旧的) 默认情况下,OSD守护进程每半秒钟便会向Monitor报告其PG等相关状态,如果某个PG的主OSD所在acting set没能向Monitor发送报告,或者其他的Monitor已经报告该OSD为down时,该PG便会被标记为stale。</li>
</ol>
<h3 id="OSD状态"><a href="#OSD状态" class="headerlink" title="OSD状态"></a>OSD状态</h3><p>单个OSD有两组状态需要关注,其中一组使用in/out标记该OSD是否在集群内,另一组使用up/down标记该OSD是否处于运行中状态。两组状态之间并不互斥,换句话说,当一个OSD处于“in”状态时,它仍然可以处于up或down的状态。</p>
<ol>
<li>OSD状态为in且up 这是一个OSD正常的状态,说明该OSD处于集群内,并且运行正常。</li>
<li>OSD状态为in且down 此时该OSD尚处于集群中,但是守护进程状态已经不正常,默认在300秒后会被踢出集群,状态进而变为out且down,之后处于该OSD上的PG会迁移至其它OSD。</li>
<li>OSD状态为out且up 这种状态一般会出现在新增OSD时,意味着该OSD守护进程正常,但是尚未加入集群。</li>
<li>OSD状态为out且down 在该状态下的OSD不在集群内,并且守护进程运行不正常,CRUSH不会再分配PG到该OSD上。</li>
</ol>
<h3 id="集群监控管理"><a href="#集群监控管理" class="headerlink" title="集群监控管理"></a>集群监控管理</h3><p>集群整体运行状态</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"># ceph -s </span><br><span class="line">cluster:</span><br><span class="line">    id:8230a918-a0de-4784-9ab8-cd2a2b8671d0</span><br><span class="line">    health: HEALTH_WARN</span><br><span class="line">            application not enabled on 1 pool(s)</span><br><span class="line"></span><br><span class="line">  services:</span><br><span class="line">    mon:3 daemons, quorum cephnode01,cephnode02,cephnode03 (age 27h)</span><br><span class="line">    mgr: cephnode01(active, since 53m), standbys: cephnode03, cephnode02</span><br><span class="line">    osd:4 osds:4 up (since 27h),4in(since 19h)</span><br><span class="line">    rgw:1 daemon active (cephnode01)</span><br><span class="line"></span><br><span class="line">  data:</span><br><span class="line">    pools:6 pools,96 pgs</span><br><span class="line">    objects:235 objects,3.6KiB</span><br><span class="line">    usage:4.0GiB used,56GiB&#x2F;60GiB avail</span><br><span class="line">    pgs:96 active+clean</span><br></pre></td></tr></table></figure>

<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">id：集群ID</span><br><span class="line">health：集群运行状态，这里有一个警告，说明是有问题，意思是pg数大于pgp数，通常此数值相等。</span><br><span class="line">mon：Monitors运行状态。</span><br><span class="line">osd：OSDs运行状态。</span><br><span class="line">mgr：Managers运行状态。</span><br><span class="line">mds：Metadatas运行状态。</span><br><span class="line">pools：存储池与PGs的数量。</span><br><span class="line">objects：存储对象的数量。</span><br><span class="line">usage：存储的理论用量。</span><br><span class="line">pgs：PGs的运行状态</span><br></pre></td></tr></table></figure>

<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">~]$ ceph -w</span><br><span class="line">~]$ ceph health detail</span><br></pre></td></tr></table></figure>

<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">pg状态</span><br><span class="line">~]$ ceph pg dump</span><br><span class="line">~]$ ceph pg stat</span><br><span class="line"></span><br><span class="line">pool状态</span><br><span class="line">~]$ ceph osd pool stats</span><br><span class="line">~]$ ceph osd pool stats </span><br><span class="line"></span><br><span class="line">osd状态</span><br><span class="line">~]$ ceph osd stat</span><br><span class="line">~]$ ceph osd dump</span><br><span class="line">~]$ ceph osd tree</span><br><span class="line">~]$ ceph osd df</span><br><span class="line"></span><br><span class="line">Monitor状态和查看仲裁状态</span><br><span class="line">~]$ ceph mon stat</span><br><span class="line">~]$ ceph mon dump</span><br><span class="line">~]$ ceph quorum_status</span><br><span class="line"></span><br><span class="line">集群空间用量</span><br><span class="line">~]$ ceph df</span><br><span class="line">~]$ ceph df detail</span><br></pre></td></tr></table></figure>

<h3 id="集群配置管理-临时和全局，服务平滑重启"><a href="#集群配置管理-临时和全局，服务平滑重启" class="headerlink" title="集群配置管理(临时和全局，服务平滑重启)"></a>集群配置管理(临时和全局，服务平滑重启)</h3><p>有时候需要更改服务的配置，但不想重启服务，或者是临时修改。这时候就可以使用tell和daemon子命令来完成此需求。</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">查看运行配置</span><br><span class="line">命令格式：</span><br><span class="line"># ceph daemon &#123;daemon-type&#125;.&#123;id&#125; config show </span><br><span class="line"></span><br><span class="line">命令举例：</span><br><span class="line"># ceph daemon osd.0 config show</span><br></pre></td></tr></table></figure>

<h3 id="tell子命令格式"><a href="#tell子命令格式" class="headerlink" title="tell子命令格式"></a>tell子命令格式</h3><p>使用 tell 的方式适合对整个集群进行设置，使用 * 号进行匹配，就可以对整个集群的角色进行设置。而出现节点异常无法设置时候，只会在命令行当中进行报错，不太便于查找。</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">命令格式：</span><br><span class="line"># ceph tell &#123;daemon-type&#125;.&#123;daemon id or *&#125; injectargs --&#123;name&#125;&#x3D;&#123;value&#125; [--&#123;name&#125;&#x3D;&#123;value&#125;]</span><br><span class="line">命令举例：</span><br><span class="line"># ceph tell osd.0 injectargs --debug-osd 20 --debug-ms 1</span><br></pre></td></tr></table></figure>

<p>daemon-type：为要操作的对象类型如osd、mon、mds等。<br>daemon id：该对象的名称，osd通常为0、1等，mon为ceph -s显示的名称，这里可以输入*表示全部。<br>injectargs：表示参数注入，后面必须跟一个参数，也可以跟多个</p>
<h3 id="daemon子命令"><a href="#daemon子命令" class="headerlink" title="daemon子命令"></a>daemon子命令</h3><p>使用 daemon 进行设置的方式就是一个个的去设置，这样可以比较好的反馈，此方法是需要在设置的角色所在的主机上进行设置。</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">命令格式：</span><br><span class="line"># ceph daemon &#123;daemon-type&#125;.&#123;id&#125; config set &#123;name&#125;&#x3D;&#123;value&#125;</span><br><span class="line">命令举例：</span><br><span class="line"># ceph daemon mon.ceph-monitor-1 config set mon_allow_pool_delete false</span><br></pre></td></tr></table></figure>

<h3 id="集群操作"><a href="#集群操作" class="headerlink" title="集群操作"></a>集群操作</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">1、启动所有守护进程</span><br><span class="line"># systemctl start ceph.target</span><br><span class="line">2、按类型启动守护进程</span><br><span class="line"># systemctl start ceph-mgr.target</span><br><span class="line"># systemctl start ceph-osd@id</span><br><span class="line"># systemctl start ceph-mon.target</span><br><span class="line"># systemctl start ceph-mds.target</span><br><span class="line"># systemctl start ceph-radosgw.target</span><br></pre></td></tr></table></figure>

<h3 id="添加和删除osd"><a href="#添加和删除osd" class="headerlink" title="添加和删除osd"></a>添加和删除osd</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">添加</span><br><span class="line">1、格式化磁盘</span><br><span class="line">ceph-volume lvm zap &#x2F;dev&#x2F;sd&lt;id&gt;</span><br><span class="line">2、进入到ceph-deploy执行目录&#x2F;my-cluster，添加OSD</span><br><span class="line"># ceph-deploy osd create --data &#x2F;dev&#x2F;sd&lt;id&gt; $hostname</span><br><span class="line"></span><br><span class="line">删除</span><br><span class="line">1、调整osd的crush weight为0</span><br><span class="line">ceph osd crush reweight osd.&lt;ID&gt;0.0</span><br><span class="line">2、将osd进程stop</span><br><span class="line">systemctl stop ceph-osd@&lt;ID&gt;</span><br><span class="line">3、将osd设置out</span><br><span class="line">ceph osd out&lt;ID&gt;</span><br><span class="line">4、立即执行删除OSD中数据</span><br><span class="line">ceph osd purge osd.&lt;ID&gt;--yes-i-really-mean-it</span><br><span class="line">5、卸载磁盘</span><br><span class="line">umount &#x2F;var&#x2F;lib&#x2F;ceph&#x2F;osd&#x2F;ceph-？</span><br></pre></td></tr></table></figure>

<h3 id="扩容PG"><a href="#扩容PG" class="headerlink" title="扩容PG"></a>扩容PG</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">ceph osd pool set&#123;pool-name&#125; pg_num 128</span><br><span class="line">ceph osd pool set&#123;pool-name&#125; pgp_num 128 </span><br></pre></td></tr></table></figure>

<p>注：</p>
<p>1、扩容大小取跟它接近的2的N次方。</p>
<p>2、在更改pool的PG数量时，需同时更改PGP的数量。PGP是为了管理placement而存在的专门的PG，它和PG的数量应该保持一致。如果你增加pool的pg_num，就需要同时增加pgp_num，保持它们大小一致，这样集群才能正常rebalancing。</p>
<h3 id="Pool操作"><a href="#Pool操作" class="headerlink" title="Pool操作"></a>Pool操作</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br></pre></td><td class="code"><pre><span class="line">列出存储池</span><br><span class="line">ceph osd lspools</span><br><span class="line"></span><br><span class="line">创建存储池</span><br><span class="line">命令格式：</span><br><span class="line"># ceph osd pool create &#123;pool-name&#125; &#123;pg-num&#125; [&#123;pgp-num&#125;]</span><br><span class="line">命令举例：</span><br><span class="line"># ceph osd pool create rbd  32 32</span><br><span class="line"></span><br><span class="line">设置存储池配额</span><br><span class="line">命令格式：</span><br><span class="line"># ceph osd pool set-quota &#123;pool-name&#125; [max_objects &#123;obj-count&#125;] [max_bytes &#123;bytes&#125;]</span><br><span class="line">命令举例：</span><br><span class="line"># ceph osd pool set-quota rbd max_objects 10000</span><br><span class="line"></span><br><span class="line">删除存储池</span><br><span class="line">ceph osd pool delete&#123;pool-name&#125;[&#123;pool-name&#125;--yes-i-really-really-mean-it]</span><br><span class="line"></span><br><span class="line">重命名存储池</span><br><span class="line">ceph osd pool rename &#123;current-pool-name&#125;&#123;new-pool-name&#125;</span><br><span class="line"></span><br><span class="line">查看存储池统计信息</span><br><span class="line">rados df</span><br><span class="line"></span><br><span class="line">给存储池做快照</span><br><span class="line">ceph osd pool mksnap &#123;pool-name&#125;&#123;snap-name&#125;</span><br><span class="line"></span><br><span class="line">删除存储池的快照</span><br><span class="line">ceph osd pool rmsnap &#123;pool-name&#125;&#123;snap-name&#125;</span><br><span class="line"></span><br><span class="line">获取存储池选项值</span><br><span class="line">ceph osd pool get&#123;pool-name&#125;&#123;key&#125;</span><br><span class="line"></span><br><span class="line">调整存储池选项值</span><br><span class="line">ceph osd pool set&#123;pool-name&#125;&#123;key&#125;&#123;value&#125;</span><br><span class="line">size：设置存储池中的对象副本数，详情参见设置对象副本数。仅适用于副本存储池。</span><br><span class="line">min_size：设置 I&#x2F;O 需要的最小副本数，详情参见设置对象副本数。仅适用于副本存储池。</span><br><span class="line">pg_num：计算数据分布时的有效 PG 数。只能大于当前 PG 数。</span><br><span class="line">pgp_num：计算数据分布时使用的有效 PGP 数量。小于等于存储池的 PG 数。</span><br><span class="line">hashpspool：给指定存储池设置&#x2F;取消 HASHPSPOOL 标志。</span><br><span class="line">target_max_bytes：达到 max_bytes 阀值时会触发Ceph冲洗或驱逐对象。</span><br><span class="line">target_max_objects：达到 max_objects 阀值时会触发Ceph冲洗或驱逐对象。</span><br><span class="line">scrub_min_interval：在负载低时，洗刷存储池的最小间隔秒数。如果是0，就按照配置文件里的 osd_scrub_min_interval 。</span><br><span class="line">scrub_max_interval：不管集群负载如何，都要洗刷存储池的最大间隔秒数。如果是0，就按照配置文件里的 osd_scrub_max_interval 。</span><br><span class="line">deep_scrub_interval：“深度”洗刷存储池的间隔秒数。如果是0，就按照配置文件里的 osd_deep_scrub_interval 。</span><br><span class="line"></span><br><span class="line">获取对象副本数</span><br><span class="line">ceph osd dump| grep &#39;replicated size&#39;</span><br></pre></td></tr></table></figure>

<h3 id="用户管理"><a href="#用户管理" class="headerlink" title="用户管理"></a>用户管理</h3><p>Ceph 把数据以对象的形式存于各存储池中。Ceph 用户必须具有访问存储池的权限才能够读写数据。另外，Ceph 用户必须具有执行权限才能够使用 Ceph 的管理命令。</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">查看用户信息</span><br><span class="line">查看所有用户信息</span><br><span class="line"># ceph auth list</span><br><span class="line">获取所有用户的key与权限相关信息</span><br><span class="line"># ceph auth get client.admin</span><br><span class="line">如果只需要某个用户的key信息，可以使用pring-key子命令</span><br><span class="line"># ceph auth print-key client.admin </span><br><span class="line"></span><br><span class="line">添加用户</span><br><span class="line"># ceph auth add client.john mon &#39;allow r&#39; osd &#39;allow rw pool&#x3D;liverpool&#39;</span><br><span class="line"># ceph auth get-or-create client.paul mon &#39;allow r&#39; osd &#39;allow rw pool&#x3D;liverpool&#39;</span><br><span class="line"># ceph auth get-or-create client.george mon &#39;allow r&#39; osd &#39;allow rw pool&#x3D;liverpool&#39; -o george.keyring</span><br><span class="line"># ceph auth get-or-create-key client.ringo mon &#39;allow r&#39; osd &#39;allow rw pool&#x3D;liverpool&#39; -o ringo.key</span><br><span class="line"></span><br><span class="line">修改用户权限</span><br><span class="line"># ceph auth caps client.john mon &#39;allow r&#39; osd &#39;allow rw pool&#x3D;liverpool&#39;</span><br><span class="line"># ceph auth caps client.paul mon &#39;allow rw&#39; osd &#39;allow rwx pool&#x3D;liverpool&#39;</span><br><span class="line"># ceph auth caps client.brian-manager mon &#39;allow *&#39; osd &#39;allow *&#39;</span><br><span class="line"># ceph auth caps client.ringo mon &#39; &#39; osd &#39; &#39;</span><br><span class="line"></span><br><span class="line">删除用户</span><br><span class="line"># ceph auth del &#123;TYPE&#125;.&#123;ID&#125;</span><br><span class="line">其中， &#123;TYPE&#125; 是 client，osd，mon 或 mds 的其中一种。&#123;ID&#125; 是用户的名字或守护进程的 ID 。</span><br></pre></td></tr></table></figure>

<h3 id="增加和删除Monitor"><a href="#增加和删除Monitor" class="headerlink" title="增加和删除Monitor"></a>增加和删除Monitor</h3><p>一个集群可以只有一个 monitor，推荐生产环境至少部署 3 个。Ceph 使用 Paxos 算法的一个变种对各种 map 、以及其它对集群来说至关重要的信息达成共识。建议（但不是强制）部署奇数个 monitor 。Ceph 需要 mon 中的大多数在运行并能够互相通信，比如单个 mon，或 2 个中的 2 个，3 个中的 2 个，4 个中的 3 个等。初始部署时，建议部署 3 个 monitor。后续如果要增加，请一次增加 2 个。</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">新增一个monitor</span><br><span class="line"># ceph-deploy mon create $hostname</span><br><span class="line">注意：执行ceph-deploy之前要进入之前安装时候配置的目录。&#x2F;my-cluster</span><br><span class="line"></span><br><span class="line">删除monitor</span><br><span class="line"># ceph-deploy mon destroy $hostname</span><br><span class="line">注意：确保你删除某个Mon后，其余Mon仍能达成一致。如果不可能，删除它之前可能需要先增加一个。</span><br></pre></td></tr></table></figure>

<p>\END -</p>
 
      <!-- reward -->
      
      <div id="reword-out">
        <div id="reward-btn">
          打赏
        </div>
      </div>
      
    </div>
    

    <!-- copyright -->
    
    <div class="declare">
      <ul class="post-copyright">
        <li>
          <i class="ri-copyright-line"></i>
          <strong>版权声明： </strong>
          
          本博客所有文章除特别声明外，著作权归作者所有。转载请注明出处！
          
        </li>
      </ul>
    </div>
    
    <footer class="article-footer">
       
<div class="share-btn">
      <span class="share-sns share-outer">
        <i class="ri-share-forward-line"></i>
        分享
      </span>
      <div class="share-wrap">
        <i class="arrow"></i>
        <div class="share-icons">
          
          <a class="weibo share-sns" href="javascript:;" data-type="weibo">
            <i class="ri-weibo-fill"></i>
          </a>
          <a class="weixin share-sns wxFab" href="javascript:;" data-type="weixin">
            <i class="ri-wechat-fill"></i>
          </a>
          <a class="qq share-sns" href="javascript:;" data-type="qq">
            <i class="ri-qq-fill"></i>
          </a>
          <a class="douban share-sns" href="javascript:;" data-type="douban">
            <i class="ri-douban-line"></i>
          </a>
          <!-- <a class="qzone share-sns" href="javascript:;" data-type="qzone">
            <i class="icon icon-qzone"></i>
          </a> -->
          
          <a class="facebook share-sns" href="javascript:;" data-type="facebook">
            <i class="ri-facebook-circle-fill"></i>
          </a>
          <a class="twitter share-sns" href="javascript:;" data-type="twitter">
            <i class="ri-twitter-fill"></i>
          </a>
          <a class="google share-sns" href="javascript:;" data-type="google">
            <i class="ri-google-fill"></i>
          </a>
        </div>
      </div>
</div>

<div class="wx-share-modal">
    <a class="modal-close" href="javascript:;"><i class="ri-close-circle-line"></i></a>
    <p>扫一扫，分享到微信</p>
    <div class="wx-qrcode">
      <img src="//api.qrserver.com/v1/create-qr-code/?size=150x150&data=http://example.com/2020/11/11/deploy/Ceph%E5%88%86%E5%B8%83%E5%BC%8F%E5%AD%98%E5%82%A8%E6%97%A5%E5%B8%B8%E8%BF%90%E7%BB%B4%E7%AE%A1%E7%90%86/" alt="微信分享二维码">
    </div>
</div>

<div id="share-mask"></div>  
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/deploy/" rel="tag">deploy</a></li></ul>

    </footer>
  </div>

   
  <nav class="article-nav">
    
      <a href="/2020/11/11/deploy/%E5%BC%80%E6%BA%90%E3%80%81%E5%BC%BA%E5%A4%A7%E7%9A%84Linux%E6%9C%8D%E5%8A%A1%E5%99%A8%E9%9B%86%E7%BE%A4%E7%AE%A1%E7%90%86%E5%B7%A5%E5%85%B7%EF%BC%8C%E6%AF%94%E5%AE%9D%E5%A1%94%E5%A5%BD%E7%94%A8/" class="article-nav-link">
        <strong class="article-nav-caption">上一篇</strong>
        <div class="article-nav-title">
          
            开源、强大的Linux服务器集群管理工具，比宝塔好用.md
          
        </div>
      </a>
    
    
      <a href="/2020/11/11/deploy/Gitlab%E4%B8%AD%E6%96%87%E7%A4%BE%E5%8C%BA%E7%89%88%E9%83%A8%E7%BD%B2/" class="article-nav-link">
        <strong class="article-nav-caption">下一篇</strong>
        <div class="article-nav-title">Gitlab中文社区版部署.md</div>
      </a>
    
  </nav>

   
<!-- valine评论 -->
<div id="vcomments-box">
  <div id="vcomments"></div>
</div>
<script src="//cdn1.lncld.net/static/js/3.0.4/av-min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/valine@1.4.14/dist/Valine.min.js"></script>
<script>
  new Valine({
    el: "#vcomments",
    app_id: "",
    app_key: "",
    path: window.location.pathname,
    avatar: "monsterid",
    placeholder: "给我的文章加点评论吧~",
    recordIP: true,
  });
  const infoEle = document.querySelector("#vcomments .info");
  if (infoEle && infoEle.childNodes && infoEle.childNodes.length > 0) {
    infoEle.childNodes.forEach(function (item) {
      item.parentNode.removeChild(item);
    });
  }
</script>
<style>
  #vcomments-box {
    padding: 5px 30px;
  }

  @media screen and (max-width: 800px) {
    #vcomments-box {
      padding: 5px 0px;
    }
  }

  #vcomments-box #vcomments {
    background-color: #fff;
  }

  .v .vlist .vcard .vh {
    padding-right: 20px;
  }

  .v .vlist .vcard {
    padding-left: 10px;
  }
</style>

 
     
</article>

</section>
      <footer class="footer">
  <div class="outer">
    <ul>
      <li>
        Copyrights &copy;
        2015-2020
        <i class="ri-heart-fill heart_icon"></i> TzWind
      </li>
    </ul>
    <ul>
      <li>
        
        
        
        由 <a href="https://hexo.io" target="_blank">Hexo</a> 强力驱动
        <span class="division">|</span>
        主题 - <a href="https://github.com/Shen-Yu/hexo-theme-ayer" target="_blank">Ayer</a>
        
      </li>
    </ul>
    <ul>
      <li>
        
        
        <span>
  <span><i class="ri-user-3-fill"></i>访问人数:<span id="busuanzi_value_site_uv"></span></s>
  <span class="division">|</span>
  <span><i class="ri-eye-fill"></i>浏览次数:<span id="busuanzi_value_page_pv"></span></span>
</span>
        
      </li>
    </ul>
    <ul>
      
    </ul>
    <ul>
      
    </ul>
    <ul>
      <li>
        <!-- cnzz统计 -->
        
        <script type="text/javascript" src='https://s9.cnzz.com/z_stat.php?id=1278069914&amp;web_id=1278069914'></script>
        
      </li>
    </ul>
  </div>
</footer>
      <div class="float_btns">
        <div class="totop" id="totop">
  <i class="ri-arrow-up-line"></i>
</div>

<div class="todark" id="todark">
  <i class="ri-moon-line"></i>
</div>

      </div>
    </main>
    <aside class="sidebar on">
      <button class="navbar-toggle"></button>
<nav class="navbar">
  
  <div class="logo">
    <a href="/"><img src="/images/ayer-side.svg" alt="Hexo"></a>
  </div>
  
  <ul class="nav nav-main">
    
    <li class="nav-item">
      <a class="nav-item-link" href="/">主页</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/archives">归档</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/categories">分类</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/tags">标签</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" target="_blank" rel="noopener" href="http://www.baidu.com">百度</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/friends">友链</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/2019/about">关于我</a>
    </li>
    
  </ul>
</nav>
<nav class="navbar navbar-bottom">
  <ul class="nav">
    <li class="nav-item">
      
      <a class="nav-item-link nav-item-search"  title="搜索">
        <i class="ri-search-line"></i>
      </a>
      
      
      <a class="nav-item-link" target="_blank" href="/atom.xml" title="RSS Feed">
        <i class="ri-rss-line"></i>
      </a>
      
    </li>
  </ul>
</nav>
<div class="search-form-wrap">
  <div class="local-search local-search-plugin">
  <input type="search" id="local-search-input" class="local-search-input" placeholder="Search...">
  <div id="local-search-result" class="local-search-result"></div>
</div>
</div>
    </aside>
    <script>
      if (window.matchMedia("(max-width: 768px)").matches) {
        document.querySelector('.content').classList.remove('on');
        document.querySelector('.sidebar').classList.remove('on');
      }
    </script>
    <div id="mask"></div>

<!-- #reward -->
<div id="reward">
  <span class="close"><i class="ri-close-line"></i></span>
  <p class="reward-p"><i class="ri-cup-line"></i>请我喝杯咖啡吧~</p>
  <div class="reward-box">
    
    
  </div>
</div>
    
<script src="/js/jquery-2.0.3.min.js"></script>


<script src="/js/lazyload.min.js"></script>

<!-- Tocbot -->


<script src="/js/tocbot.min.js"></script>

<script>
  tocbot.init({
    tocSelector: '.tocbot',
    contentSelector: '.article-entry',
    headingSelector: 'h1, h2, h3, h4, h5, h6',
    hasInnerContainers: true,
    scrollSmooth: true,
    scrollContainer: 'main',
    positionFixedSelector: '.tocbot',
    positionFixedClass: 'is-position-fixed',
    fixedSidebarOffset: 'auto'
  });
</script>

<script src="https://cdn.jsdelivr.net/npm/jquery-modal@0.9.2/jquery.modal.min.js"></script>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/jquery-modal@0.9.2/jquery.modal.min.css">
<script src="https://cdn.jsdelivr.net/npm/justifiedGallery@3.7.0/dist/js/jquery.justifiedGallery.min.js"></script>

<script src="/dist/main.js"></script>

<!-- ImageViewer -->

<!-- Root element of PhotoSwipe. Must have class pswp. -->
<div class="pswp" tabindex="-1" role="dialog" aria-hidden="true">

    <!-- Background of PhotoSwipe. 
         It's a separate element as animating opacity is faster than rgba(). -->
    <div class="pswp__bg"></div>

    <!-- Slides wrapper with overflow:hidden. -->
    <div class="pswp__scroll-wrap">

        <!-- Container that holds slides. 
            PhotoSwipe keeps only 3 of them in the DOM to save memory.
            Don't modify these 3 pswp__item elements, data is added later on. -->
        <div class="pswp__container">
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
        </div>

        <!-- Default (PhotoSwipeUI_Default) interface on top of sliding area. Can be changed. -->
        <div class="pswp__ui pswp__ui--hidden">

            <div class="pswp__top-bar">

                <!--  Controls are self-explanatory. Order can be changed. -->

                <div class="pswp__counter"></div>

                <button class="pswp__button pswp__button--close" title="Close (Esc)"></button>

                <button class="pswp__button pswp__button--share" style="display:none" title="Share"></button>

                <button class="pswp__button pswp__button--fs" title="Toggle fullscreen"></button>

                <button class="pswp__button pswp__button--zoom" title="Zoom in/out"></button>

                <!-- Preloader demo http://codepen.io/dimsemenov/pen/yyBWoR -->
                <!-- element will get class pswp__preloader--active when preloader is running -->
                <div class="pswp__preloader">
                    <div class="pswp__preloader__icn">
                        <div class="pswp__preloader__cut">
                            <div class="pswp__preloader__donut"></div>
                        </div>
                    </div>
                </div>
            </div>

            <div class="pswp__share-modal pswp__share-modal--hidden pswp__single-tap">
                <div class="pswp__share-tooltip"></div>
            </div>

            <button class="pswp__button pswp__button--arrow--left" title="Previous (arrow left)">
            </button>

            <button class="pswp__button pswp__button--arrow--right" title="Next (arrow right)">
            </button>

            <div class="pswp__caption">
                <div class="pswp__caption__center"></div>
            </div>

        </div>

    </div>

</div>

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/default-skin/default-skin.min.css">
<script src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe-ui-default.min.js"></script>

<script>
    function viewer_init() {
        let pswpElement = document.querySelectorAll('.pswp')[0];
        let $imgArr = document.querySelectorAll(('.article-entry img:not(.reward-img)'))

        $imgArr.forEach(($em, i) => {
            $em.onclick = () => {
                // slider展开状态
                // todo: 这样不好，后面改成状态
                if (document.querySelector('.left-col.show')) return
                let items = []
                $imgArr.forEach(($em2, i2) => {
                    let img = $em2.getAttribute('data-idx', i2)
                    let src = $em2.getAttribute('data-target') || $em2.getAttribute('src')
                    let title = $em2.getAttribute('alt')
                    // 获得原图尺寸
                    const image = new Image()
                    image.src = src
                    items.push({
                        src: src,
                        w: image.width || $em2.width,
                        h: image.height || $em2.height,
                        title: title
                    })
                })
                var gallery = new PhotoSwipe(pswpElement, PhotoSwipeUI_Default, items, {
                    index: parseInt(i)
                });
                gallery.init()
            }
        })
    }
    viewer_init()
</script>

<!-- MathJax -->

<!-- Katex -->

<!-- busuanzi  -->


<script src="/js/busuanzi-2.3.pure.min.js"></script>


<!-- ClickLove -->

<!-- ClickBoom1 -->

<!-- ClickBoom2 -->

<!-- CodeCopy -->


<link rel="stylesheet" href="/css/clipboard.css">

<script src="https://cdn.jsdelivr.net/npm/clipboard@2/dist/clipboard.min.js"></script>
<script>
  function wait(callback, seconds) {
    var timelag = null;
    timelag = window.setTimeout(callback, seconds);
  }
  !function (e, t, a) {
    var initCopyCode = function(){
      var copyHtml = '';
      copyHtml += '<button class="btn-copy" data-clipboard-snippet="">';
      copyHtml += '<i class="ri-file-copy-2-line"></i><span>COPY</span>';
      copyHtml += '</button>';
      $(".highlight .code pre").before(copyHtml);
      $(".article pre code").before(copyHtml);
      var clipboard = new ClipboardJS('.btn-copy', {
        target: function(trigger) {
          return trigger.nextElementSibling;
        }
      });
      clipboard.on('success', function(e) {
        let $btn = $(e.trigger);
        $btn.addClass('copied');
        let $icon = $($btn.find('i'));
        $icon.removeClass('ri-file-copy-2-line');
        $icon.addClass('ri-checkbox-circle-line');
        let $span = $($btn.find('span'));
        $span[0].innerText = 'COPIED';
        
        wait(function () { // 等待两秒钟后恢复
          $icon.removeClass('ri-checkbox-circle-line');
          $icon.addClass('ri-file-copy-2-line');
          $span[0].innerText = 'COPY';
        }, 2000);
      });
      clipboard.on('error', function(e) {
        e.clearSelection();
        let $btn = $(e.trigger);
        $btn.addClass('copy-failed');
        let $icon = $($btn.find('i'));
        $icon.removeClass('ri-file-copy-2-line');
        $icon.addClass('ri-time-line');
        let $span = $($btn.find('span'));
        $span[0].innerText = 'COPY FAILED';
        
        wait(function () { // 等待两秒钟后恢复
          $icon.removeClass('ri-time-line');
          $icon.addClass('ri-file-copy-2-line');
          $span[0].innerText = 'COPY';
        }, 2000);
      });
    }
    initCopyCode();
  }(window, document);
</script>


<!-- CanvasBackground -->


    
  </div>
</body>

</html>